Knowledge management overview of feature selection problem in high-dimensional financial data: cooperative co-evolution and MapReduce perspectives

  • Received August 30, 2019;
    Accepted November 20, 2019;
    Published December 26, 2019
  • Author(s)
    A N M Bazlur Rashid
    ORCID ,
    Tonmoy Choudhury
    ORCID
  • DOI
    http://dx.doi.org/10.21511/ppm.17(4).2019.28
  • Article Info
    Volume 17 2019, Issue #4, pp. 340-359
  • TO CITE АНОТАЦІЯ
  • Cited by
    3 articles
  • 1041 Views
  • 125 Downloads

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License

The term “big data” characterizes the massive amounts of data generation by the advanced technologies in different domains using 4Vs – volume, velocity, variety, and veracity - to indicate the amount of data that can only be processed via computationally intensive analysis, the speed of their creation, the different types of data, and their accuracy. High-dimensional financial data, such as time-series and space-time data, contain a large number of features (variables) while having a small number of samples, which are used to measure various real-time business situations for financial organizations. Such datasets are normally noisy, and complex correlations may exist between their features, and many domains, including financial, lack the al analytic tools to mine the data for knowledge discovery because of the high-dimensionality. Feature selection is an optimization problem to find a minimal subset of relevant features that maximizes the classification accuracy and reduces the computations. Traditional statistical-based feature selection approaches are not adequate to deal with the curse of dimensionality associated with big data. Cooperative co-evolution, a meta-heuristic algorithm and a divide-and-conquer approach, decomposes high-dimensional problems into smaller sub-problems. Further, MapReduce, a programming model, offers a ready-to-use distributed, scalable, and fault-tolerant infrastructure for parallelizing the developed algorithm. This article presents a knowledge management overview of evolutionary feature selection approaches, state-of-the-art cooperative co-evolution and MapReduce-based feature selection techniques, and future research directions.

view full abstract hide full abstract
    • Figure 1. General feature selection process
    • Figure 2. Overall categories of evolutionary computation for feature selection
    • Figure 3. A general architecture of cooperative co-evolutionary algorithm
    • Figure 4. An outline of cooperative co-evolutionary algorithm
    • Figure 5. A typical MapReduce workflow shuffled list
    • Figure 6. The basic flowchart of a MapReduce model
    • Figure 7. Feature selections techniques based on MapReduce
    • Table 1. Feature selection techniques based on cooperative co-evolution
    • Table 2. Feature selection techniques based on cooperative co-evolution and MapReduce
Related Articles

  • Developing countries organizations’ readiness for Big Data analytics

    Billy Mathias Kalema , Motau Mokgadi doi: http://dx.doi.org/10.21511/ppm.15(1-1).2017.13
    Problems and Perspectives in Management Volume 15, 2017         Issue #1 (cont.)         pp. 260-270 Views: 2175 Downloads: 526 TO CITE АНОТАЦІЯ

    Regardless of the nature, size, or business sector, organizations are now collecting burgeoning various volumes of data in different formats. As much as voluminous data are necessary for organizations to draw good insights needed for making informed decisions, traditional architectures and existing infrastructures are limited in delivering fast analytical processing needed for these Big Data. For success organizations need to apply technologies and methods that could empower them to cost effectively analyze these Big Data. However, many organizations in developing countries are constrained with limited access to technology, finances, infrastructure and skilled manpower. Yet, for productive use of these technologies and methods needed for Big Data analytics, both the organizations and their workforce need to be prepared. The major objective for this study was to investigate developing countries organizations’ readiness for Big Data analytics. Data for the study were collected from a public sector in South Africa and analyzed quantitatively. Results indicated that scalability, ICT infrastructure, top management support, organization size, financial resources, culture, employees’ e-skills, organization’s customers’ and vendors are significant factors for organizations’ readiness for Big Data analytics. Likewise strategies, security and competitive pressure were found not to be significant. This study contributes to the scanty literature of Big Data analytics by providing empirical evidence of the factors that need attention when organizations are preparing for Big Data analytics.

  • Regulating Big Data effects in the European insurance market

    Donatella PorriniORCID Researcher ID doi: http://dx.doi.org/10.21511/ins.08(1).2017.01
    Insurance Markets and Companies Volume 8, 2017         Issue #1         pp. 6-15 Views: 2165 Downloads: 379 TO CITE АНОТАЦІЯ

    The article analyzes the regulatory framework in the insurance market in connection with the advent of Big Data, such as information collected from different sources that can be manipulated by new technologies. The use of Big Data offers significant opportunities to the insurance companies in terms of digitization of the distribution channels and greater knowledge of the customers, which is instrumental to a more effective identification of the individual’s risk profile, as well as improvement of the competitiveness. However, regulatory measures are needed for a proper use of Big Data in terms of respect of the individual privacy, potential discrimination and constraint on competition.

  • Big data, 4IR and electronic banking and banking systems applications in South Africa and Nigeria

    Patrick AjibadeORCID Researcher ID, Stephen M. Mutula doi: http://dx.doi.org/10.21511/bbs.15(2).2020.17
    Banks and Bank Systems Volume 15, 2020         Issue #2         pp. 187-199 Views: 1293 Downloads: 346 TO CITE АНОТАЦІЯ

    Efficient banking solutions are an integral part of the business integration of South African and Nigerian economies as the two largest economies in the continent. Security, effectiveness, and integration of banking systems are critical to the sustainable development of the African continent. Therefore, an empirical analysis of the production of research on banking services and systems was conducted. The aim of the study was to examine the robustness of the research findings on banking systems in terms of their importance for the economic sustainability of the continent in the era of the fourth industrial revolution. The study adopted a bibliometric analysis using software clusters to visualize the results. Due to higher visibility of outputs and likely citations, the results showed that the key terms from Google Scholar are ranked higher than outputs from Scopus. Main research interests were related to internet banking (f = 70), e-payment systems (f = 57), telephone banking (f = 56), automated teller machines (f = 54), and mobile banking (f = 40). The results also showed a very low research interest in the technical aspect of online banking services such as security (f = 19, TLS = 40), authentication (f = 17, TLS =33), network security (f =13, TLS = 33), computer crime (f = 16, TLS = 42), and online banking (f = 11, TLS =32). The study found there were insufficient outputs in the area of the fourth industrial revolution (4IR) and banking services in Africa. Future research trends should examine the impact of the 4IR and big data on the banking system, regional economic integration, and sustainable growth in the continent.

Download Preview
Close
Problems and Perspectives in Management
Problems and Perspectives in Management