官网

https://clickhouse.tech/

quick start

ubantu

wget https://repo.yandex.ru/clickhouse/deb/lts/main/clickhouse-common-static-dbg_20.3.9.70_amd64.deb

wget https://repo.yandex.ru/clickhouse/deb/lts/main/clickhouse-common-static_20.3.9.70_amd64.deb

wget https://repo.yandex.ru/clickhouse/deb/lts/main/clickhouse-server_20.3.9.70_all.deb

wget https://repo.yandex.ru/clickhouse/deb/lts/main/clickhouse-client_20.3.9.70_all.deb

中间有个弹出框，输入默认用户的密码

  1. # dpkg -i *.deb
  2. Selecting previously unselected package clickhouse-client.
  3. (Reading database ... 211706 files and directories currently installed.)
  4. Preparing to unpack clickhouse-client_20.3.9.70_all.deb ...
  5. Unpacking clickhouse-client (20.3.9.70) ...
  6. Selecting previously unselected package clickhouse-common-static.
  7. Preparing to unpack clickhouse-common-static_20.3.9.70_amd64.deb ...
  8. Unpacking clickhouse-common-static (20.3.9.70) ...
  9. Selecting previously unselected package clickhouse-common-static-dbg.
  10. Preparing to unpack clickhouse-common-static-dbg_20.3.9.70_amd64.deb ...
  11. Unpacking clickhouse-common-static-dbg (20.3.9.70) ...
  12. Selecting previously unselected package clickhouse-server.
  13. Preparing to unpack clickhouse-server_20.3.9.70_all.deb ...
  14. Unpacking clickhouse-server (20.3.9.70) ...
  15. Setting up clickhouse-common-static (20.3.9.70) ...
  16. Setting up clickhouse-common-static-dbg (20.3.9.70) ...
  17. Setting up clickhouse-server (20.3.9.70) ...
  18. ClickHouse init script has migrated to systemd. Please manually stop old server and restart the service: sudo killall clickhouse-server && sleep 5 && sudo service clickhouse-server restart
  19. Synchronizing state of clickhouse-server.service with SysV service script with /lib/systemd/systemd-sysv-install.
  20. Executing: /lib/systemd/systemd-sysv-install enable clickhouse-server
  21. Created symlink /etc/systemd/system/multi-user.target.wants/clickhouse-server.service → /etc/systemd/system/clickhouse-server.service.
  22. Path to data directory in /etc/clickhouse-server/config.xml: /var/lib/clickhouse/
  23. Setting up clickhouse-client (20.3.9.70) ...
  24. Processing triggers for systemd (245.4-4ubuntu3.2) ...

systemctl start clickhouse-server.service

clickhouse-client --password=rootroot

centos

进入下面目录，手工下载软件然后安装

  1. https://repo.clickhouse.tech/rpm/lts/x86_64/

    或者直接下载这四个文件
    wget https://repo.clickhouse.tech/rpm/lts/x86_64/clickhouse-client-20.3.9.70-2.noarch.rpm
    wget https://repo.clickhouse.tech/rpm/lts/x86_64/clickhouse-common-static-20.3.9.70-2.x86_64.rpm
    wget https://repo.clickhouse.tech/rpm/lts/x86_64/clickhouse-common-static-dbg-20.3.9.70-2.x86_64.rpm
    wget https://repo.clickhouse.tech/rpm/lts/x86_64/clickhouse-server-20.3.9.70-2.noarch.rpm

    安装
    rpm -ivh *.rpm
    yum localinstall -y *.rpm

    卸载
    rpm -qa |grep clickhouse
    rpm -e rpm包名

    systemctl start clickhouse-server.service
    systemctl status clickhouse-server.service
    systemctl stop clickhouse-server.service
    systemctl restart clickhouse-server.service

    客户端使用
    clickhouse-client

测试数据
https://clickhouse-datasets.s3.yandex.net/hits/tsv/hits_v1.tsv.xz
https://clickhouse-datasets.s3.yandex.net/visits/tsv/visits_v1.tsv.xz

yum install xz

xz -d hits_v1.tsv.xz
xz -d visits_v1.tsv.xz

创建数据库
clickhouse-client --query "CREATE DATABASE IF NOT EXISTS tutorial"

建表
-m 可以执行一个多行语句
clickhouse-client -m

  1. CREATE TABLE tutorial.hits_v1
  2. (
  3.     `WatchID` UInt64,
  4.     `JavaEnable` UInt8,
  5.     `Title` String,
  6.     `GoodEvent` Int16,
  7.     `EventTime` DateTime,
  8.     `EventDate` Date,
  9.     `CounterID` UInt32,
  10.     `ClientIP` UInt32,
  11.     `ClientIP6` FixedString(16),
  12.     `RegionID` UInt32,
  13.     `UserID` UInt64,
  14.     `CounterClass` Int8,
  15.     `OS` UInt8,
  16.     `UserAgent` UInt8,
  17.     `URL` String,
  18.     `Referer` String,
  19.     `URLDomain` String,
  20.     `RefererDomain` String,
  21.     `Refresh` UInt8,
  22.     `IsRobot` UInt8,
  23.     `RefererCategories` Array(UInt16),
  24.     `URLCategories` Array(UInt16),
  25.     `URLRegions` Array(UInt32),
  26.     `RefererRegions` Array(UInt32),
  27.     `ResolutionWidth` UInt16,
  28.     `ResolutionHeight` UInt16,
  29.     `ResolutionDepth` UInt8,
  30.     `FlashMajor` UInt8,
  31.     `FlashMinor` UInt8,
  32.     `FlashMinor2` String,
  33.     `NetMajor` UInt8,
  34.     `NetMinor` UInt8,
  35.     `UserAgentMajor` UInt16,
  36.     `UserAgentMinor` FixedString(2),
  37.     `CookieEnable` UInt8,
  38.     `JavascriptEnable` UInt8,
  39.     `IsMobile` UInt8,
  40.     `MobilePhone` UInt8,
  41.     `MobilePhoneModel` String,
  42.     `Params` String,
  43.     `IPNetworkID` UInt32,
  44.     `TraficSourceID` Int8,
  45.     `SearchEngineID` UInt16,
  46.     `SearchPhrase` String,
  47.     `AdvEngineID` UInt8,
  48.     `IsArtifical` UInt8,
  49.     `WindowClientWidth` UInt16,
  50.     `WindowClientHeight` UInt16,
  51.     `ClientTimeZone` Int16,
  52.     `ClientEventTime` DateTime,
  53.     `SilverlightVersion1` UInt8,
  54.     `SilverlightVersion2` UInt8,
  55.     `SilverlightVersion3` UInt32,
  56.     `SilverlightVersion4` UInt16,
  57.     `PageCharset` String,
  58.     `CodeVersion` UInt32,
  59.     `IsLink` UInt8,
  60.     `IsDownload` UInt8,
  61.     `IsNotBounce` UInt8,
  62.     `FUniqID` UInt64,
  63.     `HID` UInt32,
  64.     `IsOldCounter` UInt8,
  65.     `IsEvent` UInt8,
  66.     `IsParameter` UInt8,
  67.     `DontCountHits` UInt8,
  68.     `WithHash` UInt8,
  69.     `HitColor` FixedString(1),
  70.     `UTCEventTime` DateTime,
  71.     `Age` UInt8,
  72.     `Sex` UInt8,
  73.     `Income` UInt8,
  74.     `Interests` UInt16,
  75.     `Robotness` UInt8,
  76.     `GeneralInterests` Array(UInt16),
  77.     `RemoteIP` UInt32,
  78.     `RemoteIP6` FixedString(16),
  79.     `WindowName` Int32,
  80.     `OpenerName` Int32,
  81.     `HistoryLength` Int16,
  82.     `BrowserLanguage` FixedString(2),
  83.     `BrowserCountry` FixedString(2),
  84.     `SocialNetwork` String,
  85.     `SocialAction` String,
  86.     `HTTPError` UInt16,
  87.     `SendTiming` Int32,
  88.     `DNSTiming` Int32,
  89.     `ConnectTiming` Int32,
  90.     `ResponseStartTiming` Int32,
  91.     `ResponseEndTiming` Int32,
  92.     `FetchTiming` Int32,
  93.     `RedirectTiming` Int32,
  94.     `DOMInteractiveTiming` Int32,
  95.     `DOMContentLoadedTiming` Int32,
  96.     `DOMCompleteTiming` Int32,
  97.     `LoadEventStartTiming` Int32,
  98.     `LoadEventEndTiming` Int32,
  99.     `NSToDOMContentLoadedTiming` Int32,
  100.     `FirstPaintTiming` Int32,
  101.     `RedirectCount` Int8,
  102.     `SocialSourceNetworkID` UInt8,
  103.     `SocialSourcePage` String,
  104.     `ParamPrice` Int64,
  105.     `ParamOrderID` String,
  106.     `ParamCurrency` FixedString(3),
  107.     `ParamCurrencyID` UInt16,
  108.     `GoalsReached` Array(UInt32),
  109.     `OpenstatServiceName` String,
  110.     `OpenstatCampaignID` String,
  111.     `OpenstatAdID` String,
  112.     `OpenstatSourceID` String,
  113.     `UTMSource` String,
  114.     `UTMMedium` String,
  115.     `UTMCampaign` String,
  116.     `UTMContent` String,
  117.     `UTMTerm` String,
  118.     `FromTag` String,
  119.     `HasGCLID` UInt8,
  120.     `RefererHash` UInt64,
  121.     `URLHash` UInt64,
  122.     `CLID` UInt32,
  123.     `YCLID` UInt64,
  124.     `ShareService` String,
  125.     `ShareURL` String,
  126.     `ShareTitle` String,
  127.     `ParsedParams` Nested(
  128.         Key1 String,
  129.         Key2 String,
  130.         Key3 String,
  131.         Key4 String,
  132.         Key5 String,
  133.         ValueDouble Float64),
  134.     `IslandID` FixedString(16),
  135.     `RequestNum` UInt32,
  136.     `RequestTry` UInt8
  137. )
  138. ENGINE = MergeTree()
  139. PARTITION BY toYYYYMM(EventDate)
  140. ORDER BY (CounterID, EventDate, intHash32(UserID))
  141. SAMPLE BY intHash32(UserID)
  142. SETTINGS index_granularity = 8192;

存储引擎 MergeTree

按EventData分区

按CounterID, EventDate, intHash32(UserID)三列的顺序存储数据，如果表中不显式指定主键，则以此为默认主键

按intHash32(UserID)采样

index_granularity设置索引粒度

  1. CREATE TABLE tutorial.visits_v1
  2. (
  3.     `CounterID` UInt32,
  4.     `StartDate` Date,
  5.     `Sign` Int8,
  6.     `IsNew` UInt8,
  7.     `VisitID` UInt64,
  8.     `UserID` UInt64,
  9.     `StartTime` DateTime,
  10.     `Duration` UInt32,
  11.     `UTCStartTime` DateTime,
  12.     `PageViews` Int32,
  13.     `Hits` Int32,
  14.     `IsBounce` UInt8,
  15.     `Referer` String,
  16.     `StartURL` String,
  17.     `RefererDomain` String,
  18.     `StartURLDomain` String,
  19.     `EndURL` String,
  20.     `LinkURL` String,
  21.     `IsDownload` UInt8,
  22.     `TraficSourceID` Int8,
  23.     `SearchEngineID` UInt16,
  24.     `SearchPhrase` String,
  25.     `AdvEngineID` UInt8,
  26.     `PlaceID` Int32,
  27.     `RefererCategories` Array(UInt16),
  28.     `URLCategories` Array(UInt16),
  29.     `URLRegions` Array(UInt32),
  30.     `RefererRegions` Array(UInt32),
  31.     `IsYandex` UInt8,
  32.     `GoalReachesDepth` Int32,
  33.     `GoalReachesURL` Int32,
  34.     `GoalReachesAny` Int32,
  35.     `SocialSourceNetworkID` UInt8,
  36.     `SocialSourcePage` String,
  37.     `MobilePhoneModel` String,
  38.     `ClientEventTime` DateTime,
  39.     `RegionID` UInt32,
  40.     `ClientIP` UInt32,
  41.     `ClientIP6` FixedString(16),
  42.     `RemoteIP` UInt32,
  43.     `RemoteIP6` FixedString(16),
  44.     `IPNetworkID` UInt32,
  45.     `SilverlightVersion3` UInt32,
  46.     `CodeVersion` UInt32,
  47.     `ResolutionWidth` UInt16,
  48.     `ResolutionHeight` UInt16,
  49.     `UserAgentMajor` UInt16,
  50.     `UserAgentMinor` UInt16,
  51.     `WindowClientWidth` UInt16,
  52.     `WindowClientHeight` UInt16,
  53.     `SilverlightVersion2` UInt8,
  54.     `SilverlightVersion4` UInt16,
  55.     `FlashVersion3` UInt16,
  56.     `FlashVersion4` UInt16,
  57.     `ClientTimeZone` Int16,
  58.     `OS` UInt8,
  59.     `UserAgent` UInt8,
  60.     `ResolutionDepth` UInt8,
  61.     `FlashMajor` UInt8,
  62.     `FlashMinor` UInt8,
  63.     `NetMajor` UInt8,
  64.     `NetMinor` UInt8,
  65.     `MobilePhone` UInt8,
  66.     `SilverlightVersion1` UInt8,
  67.     `Age` UInt8,
  68.     `Sex` UInt8,
  69.     `Income` UInt8,
  70.     `JavaEnable` UInt8,
  71.     `CookieEnable` UInt8,
  72.     `JavascriptEnable` UInt8,
  73.     `IsMobile` UInt8,
  74.     `BrowserLanguage` UInt16,
  75.     `BrowserCountry` UInt16,
  76.     `Interests` UInt16,
  77.     `Robotness` UInt8,
  78.     `GeneralInterests` Array(UInt16),
  79.     `Params` Array(String),
  80.     `Goals` Nested(
  81.         ID UInt32,
  82.         Serial UInt32,
  83.         EventTime DateTime,
  84.         Price Int64,
  85.         OrderID String,
  86.         CurrencyID UInt32),
  87.     `WatchIDs` Array(UInt64),
  88.     `ParamSumPrice` Int64,
  89.     `ParamCurrency` FixedString(3),
  90.     `ParamCurrencyID` UInt16,
  91.     `ClickLogID` UInt64,
  92.     `ClickEventID` Int32,
  93.     `ClickGoodEvent` Int32,
  94.     `ClickEventTime` DateTime,
  95.     `ClickPriorityID` Int32,
  96.     `ClickPhraseID` Int32,
  97.     `ClickPageID` Int32,
  98.     `ClickPlaceID` Int32,
  99.     `ClickTypeID` Int32,
  100.     `ClickResourceID` Int32,
  101.     `ClickCost` UInt32,
  102.     `ClickClientIP` UInt32,
  103.     `ClickDomainID` UInt32,
  104.     `ClickURL` String,
  105.     `ClickAttempt` UInt8,
  106.     `ClickOrderID` UInt32,
  107.     `ClickBannerID` UInt32,
  108.     `ClickMarketCategoryID` UInt32,
  109.     `ClickMarketPP` UInt32,
  110.     `ClickMarketCategoryName` String,
  111.     `ClickMarketPPName` String,
  112.     `ClickAWAPSCampaignName` String,
  113.     `ClickPageName` String,
  114.     `ClickTargetType` UInt16,
  115.     `ClickTargetPhraseID` UInt64,
  116.     `ClickContextType` UInt8,
  117.     `ClickSelectType` Int8,
  118.     `ClickOptions` String,
  119.     `ClickGroupBannerID` Int32,
  120.     `OpenstatServiceName` String,
  121.     `OpenstatCampaignID` String,
  122.     `OpenstatAdID` String,
  123.     `OpenstatSourceID` String,
  124.     `UTMSource` String,
  125.     `UTMMedium` String,
  126.     `UTMCampaign` String,
  127.     `UTMContent` String,
  128.     `UTMTerm` String,
  129.     `FromTag` String,
  130.     `HasGCLID` UInt8,
  131.     `FirstVisit` DateTime,
  132.     `PredLastVisit` Date,
  133.     `LastVisit` Date,
  134.     `TotalVisits` UInt32,
  135.     `TraficSource` Nested(
  136.         ID Int8,
  137.         SearchEngineID UInt16,
  138.         AdvEngineID UInt8,
  139.         PlaceID UInt16,
  140.         SocialSourceNetworkID UInt8,
  141.         Domain String,
  142.         SearchPhrase String,
  143.         SocialSourcePage String),
  144.     `Attendance` FixedString(16),
  145.     `CLID` UInt32,
  146.     `YCLID` UInt64,
  147.     `NormalizedRefererHash` UInt64,
  148.     `SearchPhraseHash` UInt64,
  149.     `RefererDomainHash` UInt64,
  150.     `NormalizedStartURLHash` UInt64,
  151.     `StartURLDomainHash` UInt64,
  152.     `NormalizedEndURLHash` UInt64,
  153.     `TopLevelDomain` UInt64,
  154.     `URLScheme` UInt64,
  155.     `OpenstatServiceNameHash` UInt64,
  156.     `OpenstatCampaignIDHash` UInt64,
  157.     `OpenstatAdIDHash` UInt64,
  158.     `OpenstatSourceIDHash` UInt64,
  159.     `UTMSourceHash` UInt64,
  160.     `UTMMediumHash` UInt64,
  161.     `UTMCampaignHash` UInt64,
  162.     `UTMContentHash` UInt64,
  163.     `UTMTermHash` UInt64,
  164.     `FromHash` UInt64,
  165.     `WebVisorEnabled` UInt8,
  166.     `WebVisorActivity` UInt32,
  167.     `ParsedParams` Nested(
  168.         Key1 String,
  169.         Key2 String,
  170.         Key3 String,
  171.         Key4 String,
  172.         Key5 String,
  173.         ValueDouble Float64),
  174.     `Market` Nested(
  175.         Type UInt8,
  176.         GoalID UInt32,
  177.         OrderID String,
  178.         OrderPrice Int64,
  179.         PP UInt32,
  180.         DirectPlaceID UInt32,
  181.         DirectOrderID UInt32,
  182.         DirectBannerID UInt32,
  183.         GoodID String,
  184.         GoodName String,
  185.         GoodQuantity Int32,
  186.         GoodPrice Int64),
  187.     `IslandID` FixedString(16)
  188. )
  189. ENGINE = CollapsingMergeTree(Sign)
  190. PARTITION BY toYYYYMM(StartDate)
  191. ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID)
  192. SAMPLE BY intHash32(UserID)
  193. SETTINGS index_granularity = 8192;
  1. ch2 :) use tutorial;
  2.  
  3. USE tutorial
  4.  
  5. Ok.
  6.  
  7. 0 rows in set. Elapsed: 0.002 sec. 
  8.  
  9. ch2 :) show tables;
  10.  
  11. SHOW TABLES
  12.  
  13. ┌─name──────┐
  14. │ hits_v1   │
  15. │ visits_v1 │
  16. └───────────┘
  17.  
  18. 2 rows in set. Elapsed: 0.004 sec.

数据导入

  1. date
  2. clickhouse-client --query "INSERT INTO tutorial.hits_v1 FORMAT TSV" --max_insert_block_size=100000 < hits_v1.tsv
  3. date

FORMAT TSV 导入的数据格式为TSV

max_insert_block_size 一次导入10万条记录

2.5G数据，74秒，平均 34M/S

  1. [root@ch2 data]# date
  2. Fri Jul 31 22:09:16 EDT 2020
  3. TSV" --max_insert_block_size=100000 < hits_v1.tsvT INTO tutorial.hits_v1 FORMAT  
  4.  
  5. [root@ch2 data]# date
  6. Fri Jul 31 22:10:28 EDT 2020
  7. [root@ch2 data]#
  8. [root@ch2 data]# ls -ltrh
  9. total 9.8G
  10. -rw-r--r-- 1 root root 7.3G Dec 27  2018 hits_v1.tsv
  11. -rw-r--r-- 1 root root 2.5G Dec 27  2018 visits_v1.tsv

7.3G数据，26秒，285M/S

  1. [root@ch2 data]# date
  2. Fri Jul 31 22:14:04 EDT 2020
  3.  
  4. [root@ch2 data]# date
  5. Fri Jul 31 22:14:30 EDT 2020
  1. ch2 :) use tutorial;
  2.  
  3. USE tutorial
  4.  
  5. Ok.
  6.  
  7. 0 rows in set. Elapsed: 0.002 sec. 
  8.  
  9. ch2 :) show tables;
  10.  
  11. SHOW TABLES
  12.  
  13. ┌─name──────┐
  14. │ hits_v1   │
  15. │ visits_v1 │
  16. └───────────┘
  17.  
  18. 2 rows in set. Elapsed: 0.003 sec. 
  19.  
  20. ch2 :) select count() from hits_v1;
  21.  
  22. SELECT count()
  23. FROM hits_v1
  24.  
  25. ┌─count()─┐
  26. │ 8873898 │
  27. └─────────┘
  28.  
  29. 1 rows in set. Elapsed: 0.005 sec. 
  30.  
  31. ch2 :) select count() from visits_v1;
  32.  
  33. SELECT count()
  34. FROM visits_v1
  35.  
  36. ┌─count()─┐
  37. │ 1681989 │
  38. └─────────┘
  39.  
  40. 1 rows in set. Elapsed: 0.004 sec.

 查询

  1. SELECT
  2.     StartURL AS URL,
  3.     AVG(Duration) AS AvgDuration
  4. FROM tutorial.visits_v1
  5. WHERE StartDate BETWEEN '2014-03-23' AND '2014-03-30'
  6. GROUP BY URL
  7. ORDER BY AvgDuration DESC
  8. LIMIT 10;
  1. SELECT
  2.     StartURL AS URL,
  3.     AVG(Duration) AS AvgDuration
  4. FROM tutorial.visits_v1
  5. WHERE (StartDate >= '2014-03-23') AND (StartDate <= '2014-03-30')
  6. GROUP BY URL
  7. ORDER BY AvgDuration DESC
  8. LIMIT 10
  9.  
  10. ┌─URL─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬────────AvgDuration─┐
  11. │ http://itpalanija-pri-patrivative=0&ads_app_user                                                                                                                                                │              60127 │
  12. │ http://renaul-myd-ukraine                                                                                                                                                                       │              58938 │
  13. │ http://e.mail=on&default?abid=2061&scd=yes&option?r=city_inter.com/menu&site-zaferio.ru/c/m.ensor.net/ru/login=false&orderStage.php?Brandidatamalystyle/20Mar2014%2F007%2F94dc8d2e06e56ed56bbdd │              51378 │
  14. │ https://moda/vyikrorable.com/notification                                                                                                                                                       │            48828.6 │
  15. │ http://karta/Futbol/dynas.com/haberler.ru/messages.yandsearchives/494503_lte_13800200319                                                                                                        │ 48484.666666666664 │
  16. │ http://karta/Futbol/dynamo.kiev.ua/kawaica.su/648                                                                                                                                               │            46272.2 │
  17. │ https://moda/vyikroforum1/top.ru/moscow/delo-product/trend_sms/multitryaset/news/2014/03/201000                                                                                                 │ 41531.666666666664 │
  18. │ http:%2F%2Fallback/angleNews                                                                                                                                                                    │  38878.29268292683 │
  19. │ http://xmusic/vstreatings of speeds                                                                                                                                                             │              36925 │
  20. │ http://bashmelnykh-metode.net/video/#!/video/emberkas.ru/detskij-yazi.com/iframe/default.aspx?id=760928&noreask=1&source                                                                        │              34323 │
  21. └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────────────────┘
  22.  
  23. 10 rows in set. Elapsed: 0.147 sec. Processed 1.48 million rows, 116.66 MB (10.06 million rows/s., 793.94 MB/s.) 
  1. SELECT
  2.     sum(Sign) AS visits,
  3.     sumIf(Sign, has(Goals.ID, 1105530)) AS goal_visits,
  4.     (100. * goal_visits) / visits AS goal_percent
  5. FROM tutorial.visits_v1
  6. WHERE (CounterID = 912887) AND (toYYYYMM(StartDate) = 201403) AND (domain(StartURL) = 'yandex.ru')
  7.  
  8. ┌─visits─┬─goal_visits─┬──────goal_percent─┐
  9. │  10543 │        8553 │ 81.12491700654462 │
  10. └────────┴─────────────┴───────────────────┘
  11.  
  12. 1 rows in set. Elapsed: 0.032 sec. Processed 47.60 thousand rows, 5.39 MB (1.49 million rows/s., 169.11 MB/s.)


clickhouse安装数据导入及查询测试的更多相关文章

  1. Elasticsearch学习笔记——安装、数据导入和查询

    到elasticsearch网站下载最新版本的elasticsearch 6.2.1 ? 1 https://www.elastic.co/downloads/elasticsearch 中文文档请参 ...

  2. Elasticsearch+Mongo亿级别数据导入及查询实践

    数据方案: 在Elasticsearch中通过code及time字段查询对应doc的mongo_id字段获得mongodb中的主键_id 通过获得id再进入mongodb进行查询   1,数据情况: ...

  3. sqoop1的安装以及数据导入导出测试

    下载 wget http://mirror.bit.edu.cn/apache/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz 解压 tar -zxf ...

  4. ClickHouse学习系列之八【数据导入迁移&同步】

    背景 在介绍了一些ClickHouse相关的系列文章之后,大致对ClickHouse有了比较多的了解.它是一款非常优秀的OLAP数据库,为了更好的来展示其强大的OLAP能力,本文将介绍一些快速导入大量 ...

  5. 大数据学习day26----hive01----1hive的简介 2 hive的安装（hive的两种连接方式，后台启动，标准输出，错误输出）3. 数据库的基本操作 4. 建表（内部表和外部表的创建以及应用场景，数据导入，学生、分数sql练习）5.分区表 6加载数据的方式

    1. hive的简介(具体见文档) Hive是分析处理结构化数据的工具   本质:将hive sql转化成MapReduce程序或者spark程序 Hive处理的数据一般存储在HDFS上,其分析数据底 ...

  6. Sqoop安装配置及数据导入导出

    前置条件 已经成功安装配置Hadoop和Mysql数据库服务器,如果将数据导入或从Hbase导出,还应该已经成功安装配置Hbase. 下载sqoop和Mysql的JDBC驱动 sqoop-1.2.0- ...

  7. Linux下mongodb安装及数据导入导出教程

    Linux下mongodb安装及数据导入导出教程 #查看linux发行版本 cat /etc/issue #查看linux内核版本号 uname -r 一.Linux下mongodb安装的一般步骤 1 ...

  8. Database基础(三)：SQL数据导入/导出、 操作表记录、查询及匹配条件

    一.SQL数据导入/导出 目标: 使用SQL语句完成下列导出.导入操作: 将/etc/passwd文件导入userdb库userlist表并给每条记录加编号 将userdb库userlist表中UID ...

  9. solr6.1-----mysql 数据导入-查询

    此部分一定要细心,lz 中间错了一个细节,调了好长时间(汗).请严格按照步骤操作 新建core 步骤1: 在webapps中solrhome下新建一个文件夹名字叫做collection1(名字不固定, ...

随机推荐

  1. GDB 调试技巧（不断更新中......）

    一.break到不同类的同名函数 方法: 在函数前面加类名以及作用域运算符 eg : break A::func //break 到类A的func函数 程序如下: //gdb_test.cpp #in ...

  2. win10 python3.8 wxpython.whl 安装步骤

     wxpython是python开发常用图形用户界面(GUI)工具之一,GUI因其直观便捷,对我们提高开发效率一定帮助.这里介绍一下新版本wxPython 4.0.1的安装过程及注意事项. 第1步:下 ...

  3. 关于 better-scroll 设置了以后无法滚动或不生效的问题

    首先在mounted里面注册组件  例:let scroll = new BScroll("#commondityLeftList")   然后打印实例化对象,例:console. ...

  4. sql sever 约束

    SQLServer中有五种约束,Primary Key约束.Foreign Key约束.Unique约束.Default约束和Check约束 1.Primary Key约束在表中常有一列或多列的组合, ...

  5. Java发展的重大事故

    1990年,在Sun计算机公司中,由Patrick Naughton.Mi keSheridan 及 James Gosling领导的小组Green Team,开发出的新的程序语言,命名为0ak, 后 ...

  6. 我罗斯方块第二次作业(Player类)

    我罗斯方块第二次作业 我的任务 完成player类的编写 player类的测试 我的计划 类的设计: Player类作为一个玩家类,需要处理和玩家有关的所有信息,以及维护玩家的游戏页面map.关于玩家 ...

  7. 大数据学习——搭建第一台Hadoop主机

    类型:学习笔记 参考:尚硅谷大数据系列教程 工具准备 1.VMware 2.CentOS 7 最小安装版 3.远程工具推荐使用 FinalShell 安装系统 1.打开VMware,根据自己的情况配置 ...

  8. 说透 Docker：虚拟化

    本章内容将讲解 Docker 虚拟化.虚拟化本质.namespace.cgroups. Docker 虚拟化 关于Docker 本小节将介绍 Docker 虚拟化的一些特点. Docker 是一个开放 ...

  9. 【Java】运行时Java对象在内存中是如何存储的？

    翻译自这一篇文章 我们知道函数在内存中实现为一个活动记录的栈.我们也知道Java方法在JVM栈区中实现为一个帧栈而Java对象是在堆区进行分配的. Java对象在堆内存中是怎样的呢?一旦对象保存在内存 ...

  10. Water 2.4 发布，一站式服务治理平台

    Water(水孕育万物...) Water 为项目开发.服务治理,提供一站式解决方案(可以理解为微服务架构支持套件).基于 Solon 框架开发,并支持完整的 Solon Cloud 规范:已在生产环 ...